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ABSTRACT 



Organizational change in educational systems' research has 
provided valuable lessons that address the improvement of schools. 

School -improvement programs specifically target schools with underachieving 
and high-poverty populations of students with the intention of improving 
educational inequities. While these and other current reform initiatives have 
incorporated recommendations from research regarding the implementation of 
innovative programs, less is known about enhancing the sustainability of such 
programs after the initiatives end. To investigate the sustainability of 
reform initiatives centered in research-based practice, a study of districts 
was conducted, termed the "Enhancement Initiative, " that had initiated a 
state- sponsored reform effort 10 years prior to the investigation. It 
established seven focus areas to be addressed by a district involving a 
sample of its schools. The Enhancement Initiative promoted locally designed, 
developed, and implemented reform that focused on identifying measurable 
student learning goals, research-based and innovative instructional 
strategies, and school -community partnerships. Four districts that had 
participated in the Enhancement Initiative from 1990 to 1994 were studied 
nine years after the reform's beginning to determine whether changes that 
were implemented were sustained. District representatives were questioned 
about reform changes that had and had not been sustained, and factors that 
promoted or hindered the retention of changes made under the reform program. 
The report presents a review of the pertinent research, and then presents the 
study methodology and results of factors influencing the sustainability of 
reform. An appendix includes instruments used for case-study interviews. 
(Contains 42 references.) (DFR) 
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Sustaining Education Reform: Influential Factors 



Research of the last three decades that focused on organizational change in educational 
systems has provided valuable lessons that address the improvement of schools (McLaughlin, 1 990; 
Nunnery, 1998; Bodilly, Keltner, Purnell, Reichardt, & Schuyler, 1998). Lessons regarding the 
implementation of education reform have influenced policy in the form of programs such as the 
federal Comprehensive School Reform Demonstration (CSRD) Program, which currently provides 
funding to nearly 1 800 schools nationwide, as well as similarly structured programs directed by state 
departments of education such as those existing in Colorado, Hawaii, North Carolina, Tennessee, 
Oregon, West Virginia, and Wisconsin (U.S. Department of Education, 2000; OESE, 2000). These 
school improvement programs specifically target buildings with underachieving and high poverty 
populations of students with the intention of ameliorating educational inequities. While these and 
other current reform initiatives have incorporated recommendations from research regarding the 
implementation of innovative programs, less is known from research about enhancing the 
sustainability of such programs after the initiatives end. 

In order to investigate the sustainability of reform initiatives centered in research-based 
practices, we conducted a study of districts that had initiated a state-sponsored reform effort 1 0 years 
prior to thefnvestigation. The initiative studied, designated the Enhancement Initiative in this paper, 
was similar to CSRD in structure (see Table 1). It established seven focus areas to be addressed by 
a district involving a sample of its schools. As can be seen in Table 1, the first six of the seven 
Enhancement focus areas listed are similar in design to the first six of the nine components that 
schools engaged in CSRD are required to address. In general, the Enhancement Initiative promoted 
locally designed, developed, and implemented reform (i.e., school reform plans) focused on 
identifying measurable student learning goals (i.e., outcomes-based education), research-based and 
innovative instructional strategies, and school-community partnerships. The state provided an 
infrastructure of support that included technical assistance from regional technology centers, 
university faculty, regional school-community service centers, and the state department of education. 
A network of participating districts was also established, funded, and managed by the state 
department of education. The state initiative did not solely target Title 1 1 schools, as the CSRD 
program does, and thus the participating districts in this study ranged from having 6% to 55% of 
students living in poverty (NCES, 1996). 



'“Local Education Agencies [i.e., school districts] allocate Title I funds to eligible school 
attendance areas based on the number of children from low-income families residing within the 
attendance area. A school above 50 percent poverty may use its Part A funds, along with other 
Federal, State, and local funds, to operate a schoolwide program to upgrade the instruction 
program in the whole school. Otherwise, a school operates a targeted assistance program in 
which the school identifies students who are failing, or most at risk of failing, to meet the State's 
challenging performance standards and who have the greatest need” (U.S. Department of 
Education.; 2000, November 13). 
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Table L 

Components of the Enhancement Initiative and the federal CSRD Program 



Enhancement Initiative 



CSRD Program 1 



1 Research-based instructional practices such 
cooperative learning, heterogeneous grouping 
students, active learning, etc. 



as 1 Innovative strategies and proven methods 
of for student learning, teaching, and school 
management 



2 



Development of a state-approved school reform plan 2 
that is consistent with the district’s longterm 
educational improvement plan. Activities should 
include communication of efforts with the community 
and receipt of a waiver from compliance with state 
administrative rules. 



Comprehensive design for effective school 
functioning that aligns curriculum, 
technology, and professional development 
into a schoolwide reform plan designed to 
enable all students to meet challenging 
state standards 



3 Outcomes-based education including a shared 3 
mission statement, exit outcomes, curriculum 
framework, instructional plan, and criterion-based 
assessment system 

4 Involvement of parents and community in 4 
development of the school mission, and school 
improvement as well as school involvement in 
community planning and school-community 
partnerships 

5 Collaboration with institutions at the state, regional 5 
and national levels to bring about change through 
applications of current education research 

6 Evaluation to assess change in teacher beliefs and 6 
understandings, instructional practices, and student 
learning, and the education change process more 
generally 

7 Access to, training in, and use of technological 7 
applications for instructional and managerial 
purposes 



Measurable goals for student performance 
tied to state standards and benchmarks for 
meeting the goals 



Meaningful involvement of parents and 
local community in planning and 
implementing school improvement 
activities 



High quality external technical support and 
assistance from a comprehensive school 
reform entity with expertise in schoolwide 
reform and improvement 

Evaluation of the implementation of school 
reforms and the student results achieved 



Coordination of resources (federal, state, 
local, and private) available to the school 
are utilized to support and sustain the 
school reform 



8 High quality and continuous teacher and 
staff professional development 

9 The program is supported by school 
faculty, administrators, and staff 



‘Information obtained from SEDL (1999, March 29) 

2 
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For our research, four districts that had participated in the Enhancement Initiative from 1 990 
to 1994 were studied nine years after the reform’s beginning to assess whether changes that were 
implemented were sustained. Staff that were and/or are currently instrumental in the district’s 
improvement efforts were the primary source of data for this study. More central to this paper, 
district representatives were questioned about (1) reform changes that have and have not been 
sustained, and (2) factors that promoted or hindered the sustainability of changes made under the 
reform program. Hence, this study addresses the sustainability of successfully implemented change 
in educational systems. 

It is hopeful that the results of this multiple case study project will be useful to districts and 
schools involved in implementing and sustaining locally-designed and developed comprehensive 
reform efforts. However, this study is limited in that only four reform districts from a single state 
were studied, and the investigation relied on qualitative data. These methodological constraints 
preclude our making strong and general conclusions. Therefore, although conclusions regarding 
sustaining education reform are drawn in this paper, it is likely that some important issues are 
missing because of their failure to emerge in this investigation or at these particular sites. 
Additionally, the relevance of the Findings reported here to a district or school depend on the specific 
features of that education system. Therefore, the results of the four cases studies reported here will 
not necessarily be applicable to any educational setting. In particular, the districts studied were 
located in rural or small town settings, and therefore many of the roadblocks and solutions occurring 
in these settings may not be found, for example, in urban settings. 

In this paper, we first present a review of research about sustaining change in educational 
organizations. Next, we present the study methodology and results of factors influencing the 
sustainability of reform. Finally, we develop recommendations of district and school capacities and 
external assistance that may be required to benefit from the precepts of this and similar research. 

Sustaining Education Reform 

Studies investigating sustained education reform are few in number, in part because few 
education reforms are implemented sufficiently such that continuation can be observed (Tyack & 
Cuban, 1995; Cohen & Ball, 1999). Sustained reform is most often defined as a continuation of 
classroom practices or other activities that have been implemented during the reform program’s 
existence, and the decisions, actions, and policies by school and district leaders that support that 
continuation (Berman & McLaughlin, 1977, 1978; Yonezawa & Stringfield, 2000; Hargreaves & 
Fink, 2000). Institutionalization of education reform, in contrast, has been defined more specifically 
as the integration of new practices into routine activities, and having those practices survive 
organizational changes such as administrative turnover or budget cycles (Huberman & Miles, 1984; 
Yin, 1978). In comparison, the intended outcomes of innovations, such as student achievement 
results, have not been used as variables in research of reform continuation, in part because 
methodologically longitudinal studies that could use student achievement data are less frequent than 
retrospective studies (Datnow & Stringfield, 2000). Also, frequent changes in assessment measures 
used by schools, districts, and states in the past decade make valid longitudinal comparisons nearly 
impossible. 
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In this report, sustained education reform is defined as the perception by those involved in 
the education system of continued implementation and practice of a change that occurred initially 
as a consequence of a reform program. The practice would need to be continued after the reform 
program has formally ended in order for sustained change to be attributed to it. For example, if the 
practice of site-based management was implemented as part of a reform program, and informants 
report that the practice is a continuation of change implemented during the program, it is considered 
to be sustained even when the practice itself is found to have changed drastically in character. When 
using this definition of sustained reform in research, it is important to include as participants 
individuals who are familiar with the history of an education system in order to assess change that 
has been sustained over several years. 

The following factors have been found to contribute to sustained education change in two 
or more research studies (Datnow & Stringfield, 2000; Stringfield, 1998; Yonezawa & Stringfield, 
2000; Berman & McLaughlin, 1977, 1978; Hargreaves & Fink, 2000; Fullan, 1999; Huberman & 
Miles, 1984): 

a. methods or practices that teachers experienced as effective in accomplishing 
school goals; 

b. school principals who effectively promoted, supported and managed change; 

c. political support for new practices from district and, if possible, state levels; 

d. continual, high-quality professional development and/or assistance for staff; and 

e. active recruitment for highly qualified faculty. 

In addition to these general findings, Berman & McLaughlin (1977) found that reforms that 
were larger in scope were more likely to be implemented and continued, and that teacher 
involvement in decision-making from the onset of a new program contributed to continuation of 
reform. Also, these researchers found that the amount of resources (financial and material) that 
supported the reform, the specific teacher practices that comprise the program, and formal evaluation 
were less vital than other factors to the implementation and continuation of innovative programs. 

Additional important factors observed by Yonezawa and Stringfield (2000) are the alignment 
of the reform design with the local organizational context and the integration of reform structures 
into the daily lives of the school community. Stringfield (1998) reports in his research of longterm, 
high-performing schools with large populations of low SES students that the following factors were 
common: the schools had clear goals, relied on rich sources and amounts of data for monitoring 
improvement, valued reliability over efficiency or low costs, and were intensively involved in 
informal improvement efforts. Hargreaves and Fink (2000) report that succession in leadership, staff 
retention, and community support also influence the continuation of innovative change. 

Additional factors that research has identified as contributing to the continuation of reform 
include the restructuring of resources to support the main goals of the district and school and ongoing 



evaluation or monitoring of success (Elmore & Burney, 1998; Klein, Medrich, & Perez-Ferreiro, 
1996). (This last factor contradicts the earlier finding of Berman and McLaughlin [1977] that 
evaluation was not an important factor for implementation and continuation of program practices.) 

Other research has focused on strategies and capacities that support ongoing school 
improvement, rather than sustaining specific practices. Hopkins (2000) and others claim that 
effective school improvement strategies are dependent in part on a school’s current performance. 
For example, he suggests that a successful school that desires continued success is likely to depend 
less on external support than on exposure to new ideas and practices, open discussion of values, 
within school collaboration, and external collaboration with consortia-type arrangements. If schools 
at different degrees of effectiveness require different strategies for ongoing improvement, it can be 
argued that the goal of education reform is not simply the implementation and continuation of 
effective practices, but also enhanced capacity for ongoing school improvement. Continual 
improvement need not imply a revolving door of school programs and practices, but rather can mean 
the continuation of effective practices and programs accompanied by the elimination and innovation 
of practices in areas where a school is ineffective (Tyack & Cuban, 1995). 

Stoll (1999) presents a model of school capacity for lasting improvement that is comprised 
of capacities organized around the following three areas: 1 ) individual teachers as learners (including 
capacities such as self-efficacy, motivation to learn, knowledge, and experience), 2) the school 
learning context (including school culture, structures, and leadership), and 3) the external context 
(including community, professional learning context, and political context). Stoll additionally posits 
strategies for developing these capacities from within a school and through external support to 
schools. From within, schools can build capacity for ongoing improvement by implementing a 
systems perspective among staff, high expectations for students, employees who are learning experts 
and staff who understand the change process, modifiable structures, and a broad definition of school 
leadership. Stoll’s recommendations for those attempting to develop school capacity from outside 
are respecting teachers as professionals, supporting ongoing professional development, helping 
schools interpret and use data, being critical friends, and supporting education in a wider social 
context (for example, in social and health programs as well as schools). 

We return to this research of sustaining educational change and strategies and capacities for 
ongoing school improvement in the results and conclusion sections of this paper. Next, the methods 
of the current study are presented. 



Methodology 



Enhancement Initiative 



The Enhancement Initiative was a three-year program - funded through state legislation and 
operated by the state department of education from 1990 to 1994 — designed by an education 
committee appointed by the midwestem state’s governor. This committee proposed the initiative as 
a response to current state challenges that included outmigration, delivery of rural education, changes 
in student populations, a desire to be a competitive location for economic growth, increasing learning 
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needs of adults, and the increasing public demand for accountability. The committee’s proposed 
approach to helping schools meet these demands was to provide incentives and support for locally 
driven change. In general, the initiative’s goals were stated as the three R’s: Revitalizing instruction, 
Reaching individual student achievement, and Relinking schools and communities. Participating 
districts received extensive technical assistance in designing and implementing outcomes based 
education, instructional methods, curriculum development, educational technology, and school- 
community partnerships. In general, the evaluation of the initiative documented the following 
changes across sites: 1) increased cross-disciplinary units and classes, 2) increased authentic 
instruction based on real world, culturally diverse, and community-based issues, and 3) curriculum 
more likely to relate to specified expected student outcomes (Cicchinelli & Keller, 1994; 
Cicchinelli, Keller, Stiller, & Thomas, 1995). 

District Sample 

Four of the eight districts that had participated in the Enhancement Initiative for all three 
years of the program were selected for this study based on the following three criteria: only districts 
that had successfully implemented Enhancement based on the initiative’s evaluation reports were 
considered (Cicchinelli, Keller, Stiller, & Thomas, 1995); the ability to find a similar comparison 
district within the state was considered; and a variety of districts in terms of location within the state 
and student demographics were sought. Based on these criteria and a desire to identify a manageable 
number of participating districts, four districts were selected out of the eight that had participated 
for all three years of the initiative. Table 2 contains demographic information for the selected 
Enhancement districts. 

An additional four comparison districts that had not participated in the Enhancement 
Initiative were selected in order to determine whether sustained changes could be attributed to 
Enhancement or were instead natural outgrowths of the educational progress in the state and nation 
overall. Comparison districts were individually matched to the Enhancement districts based on the 
following characteristics: geographic setting of district (i.e., rural vs. small town), number of students 
served, minority composition, proximity to the Enhancement district, median household income, and 
percentage of population in poverty, as determined from national and state data sources (NCES, 
1996; Quality Education Data, 1997). 

Each Enhancement district was asked to select one of their elementary schools to be included 
in the study that had been actively involved in the initiative and that would be most likely to exhibit 
remnants of program’s changes. Comparison school districts were asked to chose the elementary 
school that they believed was “furthest along” in terms of education improvement. For each selected 
school, the principal and a group of three or more teachers were interviewed. District superintendents 
and a second district administrator were also interviewed, individually or together. The original 
Enhancement site coordinator at each of these districts and two individuals from the state department 
were also interviewed. 
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Table 2. 

Demographics of Districts in Study 
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Data Collection 



First, interviews of two state officials familiar with the Enhancement Initiative and districts’ 
work since the initiative were conducted to assess these individuals’ perceptions of the long-term 
influence of the initiative, the state’s current reform activities, and to obtain their judgments of the 
eight districts in this study. Second, the Enhancement Initiative site coordinator from each of the 
Enhancement districts was interviewed to assess changes made during the reform and impressions 
about what changes were sustained and what factors played a role in maintaining each change. Third, 
the status of each district’s improvement effort was assessed through two interviews with district 
office staff: the superintendent and one other district administrator. School-level reform was assessed 
through an interview with the elementary school’s principal. Separate interview protocols were 
constructed for state department representatives, Enhancement site coordinators, superintendents, 
district administrators, and principals. A protocol for a focus group of school staff members was also 
constructed, and three to six teachers participated in this activity by addressing questions about school 
culture, instructional issues, professional development experiences, leadership opportunities, and 
community involvement. The instruments used appear in the Appendix. 

Analysis 

Interviews were recorded and transcribed, and the transcriptions were checked for 
inaccuracies. Two or three researchers reviewed each Enhancement district transcript, using 
guidelines for qualitative data analysis developed by the primary researcher (Miles & Huberman, 
1994; Erickson, 1986; Strauss & Corbin, 1990). Changes in policies, structures, and practices 
implemented during the Enhancement Initiative and factors influencing the sustainability of changes 
were coded. The reviewers discussed the coding according to identified categories. Consensus was 
reached regarding the identification of Enhancement changes and factors that influenced the 
continuation of those changes. Coders also judged whether the changes had been sustained beyond 
the end of the Enhancement Initiative. Discussion of these judgments continued until consensus was 
reached. Table 3 contains the identified elements and reviewers’ judgments regarding continuation 
of those elements for each district in which they were reported. Reform elements that were reported 
by respondents in two or more districts are included in Table 3. 

Reviewers next identified and categorized the factors reported by participants as influencing 
the sustainability of education reform. The categories of factors were identified through a combination 
of review of the identified factors and consideration of “perspectives,” “lenses,” and “frames” of 
school reform presented by other researchers (House & McQuillan, 1 998; Fink, 2000). The definitions 
of the five categories used to organize the sustainability factors appear in Table 4. Again, reviewers’ 
categorizations were discussed until consensus was reached. 

Four changes were found to be sustained at each of the Enhancement districts that 
implemented the change during the initiative: innovative, research-based instructional practices; 
school-community partnerships; use of technology in instruction; and site-based management. 

' Following the analysis of the Enhancement districts, the data collected from Comparison districts was 
then analyzed for the presence of these four, consistently sustained changes. For each of the four 
changes, a list of district and school structures, policies, and practices that address the element was 
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Table 3. 

District Elements Identified at Two or More of the Four Enhancement Sites and Current 
Status 



Element 


Definition 


L 


w 


SB 


S 


Innovative 

instructional 

practices 


Research-based, innovative curricula and/or instructional 
practices (see enhancement component 1 in Table 1) 


C 


c 


C 


c 


School- 

community 

Engagement 


Involvement of parents and community members in school 
functioning, and partnerships with community members and 
organizations (see component 4) 


C 


c 


c 


c 


Instructional 

Technology 


Dissemination of current information regarding instructional 
technology, training opportunities for staff, and use of 
technology for instruction (see component 7) 


c 


c 


c 




Site-based 

Management 


Shared decision making by consensus through an established 
committee of school staff and community members 


c 






c 


Outcomes- 

based 

education 


Outcomes-based education in which system components 
(curriculum, instruction, assessment, management, and 
professional development) are aligned with established student 
learning outcomes (see component 3) 




c 


c 


D 


Teacher 
planning time 


Time during the school day when teachers can work together 
(without students) to plan, discuss, and reflect on student 
learning and instructional practice 


c 




D 


C 


Authentic 

assessment 


Criterion-based holistic assessment practices using, for 
example, created rubrics. Also assessment based on real-world 
and relevant problems and assignments 


c 


D 


C 


D 


Networking 


Networking and collaboration with other school districts 
working on similar reform endeavors for the purpose of sharing 
information and gamering support 


c 


D 


D 


C 


Site 

coordinator 


School level position with the responsibility of facilitating the 
implementation and maintenance of school reform (usually a 
half-time position) 


c 


C 


D 


D 


District 

leadership 

committee 


District level committee(s) that address the implementation and 
maintenance of district level improvement with respect to 
district goals 




N 


R 


D 



Notes. C denotes Continued changes, D denotes Discontinued changes, R denotes changes that were 
Reinstated after discontinuation, N denotes cases in which not enough information was available to 
draw a conclusion, and an empty cell denotes cases in which a district did not report implementing 
a reform element. L = Lewis, W = Wilder, SB = Sitting Bull, and S = Sacagewea School Districts. 
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Table 4 . 

Definitions of sustainability factor categories 



Category 1: Ongoing engagement and development of capacity 

This category has to do with ongoing engagement and development of capacities attained during the reform 
initiative. For example, instructional skills acquired during the program might have become “routinely 
used” by teachers, or managerial skills such as working on a committee might continue to be used and 
enhance the committee’s functioning. The internalization of practices and continued use of practices are 
sustaining factors that fall into this category. 

Category 2: School and district culture 

Culture consists of shared behavioral regularities, norms, values, attitudes, perspectives, and relationships 
within the school and district. Activities that are not formal structures, such as maintenance of 
relationships, communication and sharing of information with staff and community members, informal 
networking and visiting with other districts, soliciting community input, valuing and soliciting teacher 
input into decisions, etc. are a part of the culture. Attitudes such as trust, a continual improvement 
perspective, innovation, valuing of community input, commitment, support of stakeholders such as 
community members are components of Culture. 

Category 3: Structures of education system 

Structures include established committees, positions, and regularly scheduled meetings such as for 
networking. Structures include resources such as allocated funding and available time for activities such 
as professional development or instructional planning. Policies for activities such as hiring, curriculum 
review, involvement of all schools, etc. Adaptation of subsequent programs to support the reform 
initiative’s goals is considered a type of structural support. 

Category 4: School and district leadership 

Leadership includes features of administrators (principals, superintendent, etc.) and school board, as well 
as relationships of these individuals or groups with others within the education system. For example, school 
board support for some policy falls under Leadership. Conflicts with the community fall under Culture, 
but conflicts of leaders with others in the education system (such as between administration and the school 
board) fall under leadership factors. 

Category 5: Political context 

This category has to do with state pressures and policies, and their influence on schools and districts. Other 
features of the political context include preservice (i.e., college and university) practices, and any other 
features of the greater context of the school and district influenced by its relevant regional, state, or national 
environment. 
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constructed for both the Enhancement and the matched Comparison districts. For each change, a 
judgment was made regarding whether the Comparison district exhibited and supported that practice 
(e.g., innovative, research-based instructional practices) to a greater, equivalent or lesser extent than 
did the Enhancement district. 



Findings 

In this section, we present the reform changes that informants reported were sustained since 
the Enhancement Initiative. Second, we present the factors that influenced the sustainability of 
change, organized in the five major categories: human capacity, culture, structure, leadership, and 
political context. 

Sustained chang es 

As indicated earlier, four changes implemented during Enhancement were found to be 
sustained in all Enhancement districts that reported implementing these elements. These sustained 
changes are teachers’ use of innovative instructional practices, engagement of schools with 
community, integration of technology into instruction, and site-based management. These sustained 
changes were central parts of this reform — that is, there was statewide technical assistance and 
infrastructure created to support implementation of these reform components — and finding that 
these elements were attributed to Enhancement and sustained is a significant outcome for this 
ambitious initiative. Table 3 reports the reform changes identified in the Enhancement districts and 
the current status of these changes. 

When matched comparison sites were examined for these four changes as described in the 
methodology section, a majority of these elements had not been implemented to the extent that they 
had been in the Enhancement districts. First, innovative, research-based practices had been 
implemented and sustained in all four of the Enhancement districts. Three of the four Comparison 
districts had not implemented innovative, research-based instructional practices and support structures 
to the extent of the matched Enhancement districts. In the fourth case, the Enhancement and 
Comparison districts were both using innovative practices and supported those practices with 
structures and resources. Second, all four Enhancement districts had implemented and sustained 
school-community engagement and collaboration. Three of the four Comparison districts had not 
implemented school-community engagement and collaboration to the same degree as the 
Enhancement districts. In the case of the fourth pair of districts, the majority of the Enhancement 
district’s constituencies were families of university students whose transiency precluded them from 
longterm, active involvement in the education system, and the Comparison district in this case, in 
contrast, had a more stable population of students and families participating in the system which 
contributed to community engagement at that site. 

Third, the use of technology in instruction was implemented and sustained at three of the 
Enhancement districts. Two of the three Comparison districts failed to demonstrate the use and 
integration of technology into instruction to a degree similar to the Enhancement districts. In the case 
of the third Comparison district, the isolation of the school district from neighboring cities was greater 



than was the case for the matched Enhancement district, and this Comparison district was investing 
in and implementing distance learning technology that the district viewed as necessary to provide 
adequate course offerings for high school students. In this case, the Comparison district’s need for 
instructional technology was probably greater than that of the similar Enhancement district. 

Fourth, site-based management had been implemented and sustained in two of the 
Enhancement districts. In one of the Comparison districts, site-based management was not 
implemented. In the case of the second Comparison district, not enough information was collected 
to definitively conclude whether or not site-based management had been implemented. 

In summary, the four areas and types of changes that had been implemented and sustained by 
Enhancement districts were less likely to be observed in the Comparison districts. A conclusion that 
the Enhancement Initiative contributed to ongoing improvement in these four areas at the 
participating districts is supported by this result. 

Elements of the Enhancement Initiative that were sustained or reinstated after discontinuation 
at only some of the four districts are the following: outcomes-based education, teacher planning time, 
authentic assessment practices, networking with other districts, a site coordinator position, and district 
leadership committees. 

Sustainability Factors 

Data were also coded for factors contributing to and detracting from the sustainability of 
reform changes. These items were organized into five general categories of school work that other 
researchers have referred to as “perspectives,” “lenses,” or “frames” of school improvement (House 
& McQuillan, 1998; Fink, 2000). For example, House and McQuillan propose that school 
improvement consists of the technological, political, and social perspectives. In Fink’s longitudinal 
study of an innovative school’s path examined the school through the lenses of context, meaning, 
leadership, structure, culture, and teachers’ work and lives. The five categories used for this analysis 
were derived from the coded sustainability factors in concert with the dimensions identified in 
previous research. These five categories of sustainability factors are the following: 

a. Ongoing engagement and development of human capacities engaged during the 
reform initiative; 

b. School and district cultures that value learning, innovation, and collaboration; 

c. District and school structures, policies, and resource allocations that support 
reform goals; 

d. Leadership of schools and district that maintains a consistent vision, a well- 
designed strategic plan, and positive relationships with members of the 
education system; and 

e. Political context demands, pressures, and supportive activities. 



In this section, each of these categories are delineated with respect to the study s data and illustrated 
with examples from the Enhancement districts. 

Ongoing engagement and development of human capacities. Human capacity for 
education reform has been defined as individuals’ knowledge and skills that address reform ideas, 
abilities to help others develop reform skills, commitment to reform, and disposition to learn (Spillane 
& Thompson, 1 997; Century, 1 999). This definition explicates that teaching or managerial knowledge 
and skill is not enough to enact or sustain reform (although this aspect of human capacity, also labeled 
the technology of education reform, is important; House & McQuillan, 1998), but that commitment 
and a disposition for learning, accompanied by the ability to develop knowledge and skills in others 
are also necessary. In this study, human capacities were mainly skills and knowledge that had been 
integrated into practice, but a flexibility in these practices that provides opportunity for continual 
improvement was also observed. 

Informants from districts reported that teachers and administrators acquired understanding and 
mastery of practices that are still very much in use because these new capacities were integrated into 
routine practice. That is, new capacities continued to be engaged after the initiative, and in some 
cases attention was paid to the professional development focused on these skills for new staff. Two 
types of new practices had been integrated at these sites: instructional methods and organizational 
governance practices. For example, the original site coordinator at Sacagewea mentioned that the 
district experienced the opportunity to see and work with different methods of teaching. These 
different instructional methods are still in place.” At Sitting Bull School District, a teacher stated that 
“there are hands-on things going on in I think every classroom” that are a result of Enhancement. 

Additionally, administrators from two sites reported that teachers have a strong sense of 
efficacy about instructing students in the state’s new standards because of teachers’ having already 
implemented a somewhat similar, outcomes-based education system during the Enhancement 
Initiative that required learning of all students to established criteria on a number of outcomes. That 
is, these districts had already mastered the practice of teaching specified outcomes to all students, and 
that experience had prepared them for the high expectations of all students that are necessary for 
teachers to have in a standards-based system. Thus, in the midst of a changing political context, 
teachers were able to sustain engagement of reform capacities at some sites. 

Organizational governance processes such as site-based management using shared decision- 
making were also a part of the Enhancement' Initiative. When these practices were integrated into 
school and district practices and policy, they continued as status quo after the initiative formally 
ended. For example, an elementary school principal from Sacagewea School District mentioned that 
teachers are very willing to be a part of the school-staff advisory team, and that some of the group 
process skills were probably implemented with Enhancement. She told us the following: 

Teachers are very open and willing to be part of a team. They like to have that 
partnership. I've just never had any difficulty with people wanting to step to the plate 
and be a part of making decisions at school.... Probably some of the training that they 
received during the [Enhancement Initiative! they sort of— helped them to understand 
what groups are all about and how they should operate and function. And people are 



much more willing to head up a committee, you know, take turns in terms of having 
to report or be the chairperson. 

At Sitting Bull School District, the original site coordinator informed us that the technology 
committee was established during Enhancement and sustained. She described the committee’s work 
in the following way: 

A district technology committee was developed [during Enhancement]. This 
committee became a powerful committee, and gained oversight of all technology 
funds. The committee did strategic planning for technology short- and long-term 
needs.. .so that such a plan was not written by any one person in isolation. We learned 
to make decisions by consensus, and that became a part of the wav we do business 
here. That process is still in place. The new superintendent is more “top down” in 
terms of the budget, but he promotes teacher leadership. 

The Sacagewea and Sitting Bull examples illustrate that governance practices can be 
developed and integrated into routine district operations in a manner that supports the continuation 
of those practices. 

As a last illustration of the importance of integrating practices, there was a case in which 
authentic assessment did not become routine and was therefore not sustained. The original site 
coordinator at Wilder School District told us the following: 

The district had began performance assessment [during Enhancement] and matching 
of student outcomes to learning objectives, funded in part by NSF. We had several 
practice sessions in which we created assessments at each grade level that were 
performance assessments that had relevance to these students [of which 90% are 
Native American]. For example, students might be asked to calculate the amount of 
fry bread ingredients needed to serve at a function with X number of guests given the 
amount of each ingredient needed for four servings. These performance assessments 
were developed by teachers, and were a staff development experience. This never 
became systematic, and would have been worthwhile. The district has multiple 
measures for their Title 1 funding, but it isn’t the same as locally developed 
performance assessments [that were used during Enhancement]. 

Other research similarly documents the importance of integrating new practices into routine 
activities for sustaining those practices (Datnow & Stringfield, 2000; Yonezawa & Stringfield, 2000). 
Indeed, some researchers suggest that unless new practices are integrated into routine activities (which 
requires support from school and district leaders as well as the school’s culture and political context, 
as are described below), these new technologies will not survive budget cycles or successions in 
leadership. 

There are factors that influenced the routinization of new practices (some of which are 
presented under separate headings below). Two of these factors are described here: observable, 
positive outcomes can sustain the support of practices and training for new staff can ameliorate the 
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negative influence of staff turnover on sustaining reform. 

Positive outcomes was one factor that supported the continuation of integrated practices. In 
particular, research has demonstrated that when practices are found by staff to be useful in meeting 
reform goals, these activities are more likely to become institutionalized and sustained than when 
activities to not promote desirable outcomes (Berman & McLaughlin, 1977, 1978; Huberman & 
Miles, 1984; Yonezawa & Stringfield, 2000). In this study, the continuation of site-based 
management at one site was attributed by the superintendent, in part, to the positive results of this 
management structure. He told us the following: 

We developed [site-based management] probably seven or eight years ago but we’ve 
sustained it because it’s worked well for us. It’s led us into some areas that we feel 
very good about - those areas being student learning, student achievement, student 
goals, kind of moving our district forward.... through that process we gained some 
direction, some new things that we could do that we took on. Some places have failed 
at that [sustaining site-based management], have gone back to a different form of 
management but it’s worked well for us here and we’ve sustained that. 

Thus, when new practices are perceived to result in positive outcomes, they are more likely to be 
supported and, hence, established as routine activities. Also, although it often requires effort to 
abandon practices that have become routinized, observable positive outcomes contribute to staff 
commitment to integrated practices rather than enticing staff to adopt novel, commercialized 
programs that are often touted to schools and districts (Fullan & Miles, 1992). 

Additionally, the training or induction of new staff is vital to sustain integration of practices 
at a school or district level when staff turnover occurs. For example, a principal at Lewis Schools 
reported the following observation regarding the professional development needs of new staff: 

Our new staff here, the ones who were not here during [Enhancement], don’t really 
understand a lot of the active teaching techniques and the focus on teaching for 
understanding. Some of our new staff are still in the mindset of how it was when they 
went to school. And so that’s the way they are teaching because they haven’t been 
taught any other way and so we need to make sure that they have a chance to learn all 
the more active ways of teaching. 

This principal reported that the district addresses this professional development need, in part, through 
a teacher who was an instructional leader during Enhancement and models effective instructional 
practices for new and veteran staff. Another principal at a different district described attentively 
familiarizing new staff with school practices such as the site-based management team: 

What I try to do with new staff, I try to bring them on with the staff advisory [team] 
right away. Because those are the [teachers] that are sort of new to the school and 
they need to hear what's been happening. And then new staff members become a part 
of how the school grows and how you - you know - progress as a school. So the new 
staff members are brought into the group process right away as a new staff member. 
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These examples highlight the importance of induction of new staff so that routinized practices and 
activities can continue at a school or district level when changes in staffing occur. 

School and district cultures that value learning, innovation and collaboration. School 
culture can be defined as the values, guiding philosophy, attitudes, norms, and relationships that exist 
in the organization (Boyd, 1992; Elmore, 2000; Fink, 2000). Traditionally in the U.S., school and 
district culture does not value focusing on instructional improvement, which makes reform a 
formidable challenge (Elmore, 2000; Cohen & Ball, 1999). In this study, sites reported that the 
Enhancement Initiative promoted districts’ and schools’ being more committed to learning, 
innovative, and collaborative, and that the resulting culture contributed to sustaining education 
reform. 



For example, Wilder School District representatives referred to a “district philosophy of 
education and theory of teaching and learning” that was implemented during and sustained since 
Enhancement. Wilder’s student education outcomes and corresponding curricula that were 
constructed during Enhancement were based on the culture of the 90% of the student population that 
is Native American. The established student learning outcomes were sustained by the district and 
adapted to conform with the state’s newer standards because the district’s philosophy includes 
teaching material in a manner that makes it relevant to students’ lives and location. Thus sustaining 
parts of the Enhancement Initiative such as a commitment to culturally relevant curricula even when 
faced with a state emphasis on learning standards was promoted by the district’s culture that included 
a commitment to a specific philosophy of education. 

At Sitting Bull Schools, the previous site coordinator told us that the most important longterm 
effect of the reform initiative was “promotion of culture and school climate for continued staff 
learning and staff growth. [Enhancement] fostered a belief that staff development is the way to 
continual improvement, and that continual improvement is valuable. That had incredible 
ramifications for our view of student improvement.” Thus, during the initiative a culture that values 
continual improvement and teacher learning was created and has contributed to the district’s 
sustaining, for example, funding for and participation in staff development. 

At a third district, one representative attributed the district’s change to block scheduling as 
a result of the district’s “continuing to look at different teaching methods” since and beginning with 
the Enhancement Initiative. Thus, these districts adopted a culture of innovation and continual 
learning (Fullan, 1999; Stoll, 1999) during Enhancement that has helped to sustain these districts’ 
continual striving to attain their reform goals, which remained fairly constant over time. 

School and district culture at some sites also included a shared focus on improved instruction 
and student learning. For example, the superintendent of Lewis School District reported a belief that 
Enhancement helped the district focus on instruction, which led to a lot of hard work from staff that 
he believes is now reflected in the district’s relatively high state assessment scores. At Sacagewea, 
a teacher (and previous site coordinator) described the most positive longterm effect of Enhancement 
as the “teachers’ individual awareness of need to change style in the classroom [in response to 
students’ needs]. ... teachers [became]more aware of individual learning styles.” 
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Collaboration and unity of staff became norms in these districts during Enhancement through 
the initiative’s promotion of the processes of decision making by consensus and common planning 
time for staff. The curriculum director at Sitting Bull Schools told us the following about 
Enhancement’s effect on staff unity: 

I think we needed, as a district, the [Enhancement Initiative] to be able to do what 
we’re doing now because of that mindset, because of that way of thinking and also 
with the [Enhancement] it really gave a sense of unity, you know, to the staff. ... We 
were looking more at this as our district and we were looking more at the big picture, 
you know, at the K- 12 picture rather than my first graders or my third graders. We had 
that sense of unity. And that’s another part that stayed with us after — after that time. 

Other writing supports this observation that ongoing collaboration that includes reflection is important 
for continual improvement (as opposed to a hopping from one reform program to another; Fullan, 
1999; Zimmerman, 1998). Furthermore, when collaboration of staff disappeared from a school or 
district’s culture (because of, for example, a change in leadership), that change created a barrier to 
maintaining a course of continual improvement. 

Collaboration with the community was also a part of these district’s and schools’ cultures. 
Districts reported an increase in valuing community representatives and organizations as a result of 
the Enhancement Initiative. A teacher at Sitting Bull Schools said that Enhancement affected “the 
way that we viewed our community and parents, and to be more inclusive. At Sacagewea Schools, 
the previous Modernization coordinator had the following to say about community involvement: 

[Enhancement] was a ‘real eye-opener’ in terms of the opportunity and awareness that 
it brought to the community about what schools are doing. Community input and 
teacher communication with the community was a result of [Enhancement]. The 
current school advisory committees, and the relationship between the schools and 
local businesses are all remnants of [Enhancement]. There is also more volunteering 
from the community in the schools. ... Relations with businesses were begun during 
[Enhancement], and there are not the same forums now as there were then for these 
partnerships. Nonetheless, there has been effort to continue this collaboration 
between school and businesses. The schools learn what’s going on in industry, and 
get answers to the questions of ‘how come I need to know this?’ 

In this example, community partnerships contributed to schools’ knowledge base regarding what 
businesses expect high school graduates to know and be able to do, and these relationships promote 
local industry’s awareness of what is happening in the schools. All of the study’s districts reported 
to us that the inclusion of and communication with community was much more valued by schools and 
districts as a result of their participation in the Enhancement Initiative, which highly promoted school- 
community partnerships. 

In one instance, changes in district and school culture that occurred subsequently to the 
Enhancement Initiative derailed the district’s reform effort. At Lewis, a new principal (who left the 
district during the year prior to our interviews) devalued the staff collegiality and collaboration that 
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were staples of the reform. We were told that reform practices are now used “behind closed doors” 
at that school and have been internalized and integrated into practice by senior staff, but that these 
practices are not shared among teachers. The principal mentioned a lack of trust among staff that 
currently prevents the continuation of staff sharing and collaborating around instruction. Under brand 
new leadership, that school is expecting to reinstate many of the practices and beliefs that were a part 
of their Enhancement program. 

Add negative influences of funding on morale of staff 

In summary, the importance of school and district culture should not be underestimated for 
sustaining progress toward reform goals after the ending of a specific reform initiative. Shared 
valuing of student learning and quality instruction, innovation, and collaboration within and outside 
of the education system were promoted by Enhancement in these districts. Furthermore, these 
changed values promoted sustaining districts’ and schools’ reform focus. 

District and school structures, policies, and resource allocations that support reform 
goals. District and school support in the form of policies, structures, and resources is crucial for 
sustaining new educational programs or practices (Bodilly et al., 1998; Glennan, 1998; Hargreaves 
& Fink, 2000; Berman & McLaughlin, 1977, 1978; Yonezaqa & Stringfield, 2000; Datnow & 
Stringfield, 2000). Furthermore, structures work in concert with culture to perpetuate reform in at 
least two ways. First, culture is formatively influenced by structures and policies (Elmore, 1996, 
2000; Cohen & Ball, 1 999). For example, districts that reserve time each week for all teachers to work 
collaboratively promote collegiality and norms of cooperation among staff members. Secondly, 
school and district policies can focus educators’ work on curriculum, instruction, and expectations 
for student learning and such explicit goals contribute to changes in teachers’ practices, skills, and 
knowledge. Some research suggests that in order for staff collaboration or professional development 
to result in positive changes in instruction, clear organizational goals that address instruction and 
student learning need to be established (Elmore, 2000). 

Districts in this study reported multiple structural and policy-related changes that they enacted 
to support sustaining reform; for example, creating new positions and restructuring responsibilities 
of existing positions, establishing new committees, modifying hiring practices, and changing funding 
allocations during or after the initiative. By paying attention to the structural needs for sustaining 
reform, these districts established new ways to support the path of continual improvement that had 
been begun during Enhancement. 

For example, in Sitting Bull Schools, the position of curriculum director was created at the 
end of Enhancement, and the roles and responsibilities of curriculum review committees were 
established during and sustained since the initiative. A staff development committee has been 
sustained since Enhancement to attend to and support teacher learning. Regarding the formal 
adoption of standards, the original site coordinator told us, “We developed district content standards 
during [Enhancement] and continue to refine those today — we consider them to be better than the 
standards that the state developed.” 
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Additionally, the flexibility of structures emerged as important at this site. Sitting Bull’s 
structures for parent involvement changed from a district-level school-community committee to 
school-level parent advisory groups, and this change was adaptive for this district’s university setting 
where the community’s mobility precludes ongoing parent involvement at the district level, while at 
the same time parents are willing to be involved in their children’s school and classroom activities. 

The creation of formal structures supporting professional development is another district 
activity that supported sustaining reform. At these sites a majority, if not all, teachers participated in 
the Enhancement Initiative’s training opportunities, partly because the initiative’s design elements 
— such as funding for substitute teachers and weekly early releases of students — supported broad- 
based participation by teachers. Some of these districts reported witnessing for the first time the value 
and impact of intensive professional development for teachers. More than one district told us that staff 
development committees were implemented and district funding for professional development was 
increased after Enhancement. Both Lewis and Sacagewea retained the structure of early release time 
for students that provided regular district-wide professional development and collaboration time for 
teachers. At Sitting Bull, the curriculum director reported that over 90% of teachers currently 
participate in after school study groups that focus on various educational issues. Thus, district 
structures supporting staff development contributed to continual learning and capacity-building in 
these systems. 

The use of resources is an important component of the federal CSRD initiative, for which the 
reallocation of resources around school goals is the intended mechanism to sustain reform after three 
years of funding (OESE, 1 998). In this study, careful allocation of resources contributed to sustaining 
reform practices. In Sitting Bull Schools, we were told that the district’s funding of staff development 
and curriculum committees (and these committees’ auspices over funds) became a part of the 
district’s operations. At Sacagewea, the district directed a new program to pick up where 
Enhancement left off. The original site coordinator told us, “Tech Prep [the new program after 
Enhancement] was the initiative in which hands-on instruction applied to the real world was 
continued, and so the momentum of [Enhancement] was not effected. The same people were 
involved in both initiatives.” In Lewis, an Annenberg Rural Challenge grant to the nearby university 
now funds activities of the same community committee that was established during and sustained 
since Enhancement. 

Structures addressing the hiring of new staff is another area attended to by districts after the 
reform ended. For example, Lewis Schools’ new high school principal was an Enhancement site 
coordinator at a neighboring district, and was hired in part because of her familiarity with the 
initiative. Regarding Sacagewea schools, a state department representative told us that “the 
community network ... recruited for a different kind of [superintendent] than they would have without 
[having participated in Enhancement]. I think [Enhancement] positioned them for real forward 
looking leadership and I also think that it made them real attractive to those kind of candidates.” 

The best example of changes in hiring policies comes from Sitting Bull, where the 
Enhancement contact person told us the following: 



ERIC 



19 



26 



The district became more aware of ways to go through the hiring process as a result of 
[Enhancement], and hiring practices were formalized in particular at every elementary school. 
Potential new staff are now interviewed by large teacher teams and by administrators and possibly 
even school board members. The interviews are geared around things valued such as interdisciplinary 
instruction and authentic learning and assessment. 

Thus, implementing a system of flexible structures, policies, and funding that address reform goals, 
in combination with the other categories of factors presented in this paper, promote the sustainability 
of reform. 

Leadership that maintains a consistent vision, a well-designed strategic plan, and 
positive relationships with members of the education system. In this study, consistency in district 
leadership emerged as an important factor in sustaining reform elements. In Sacagewea and Sitting 
Bull, new superintendents have been hired since the end of the Enhancement Initiative, but the 
constancy of other administrators was credited with continuing reform. For example, the current 
assistant superintendent at Sacagewea was involved in the initiative and is credited with sustaining 
some of the initiative’s vision. At Sitting Bull, low turnover in administration in general is believed 
to have contributed to sustaining curricular and instructional practices since Enhancement’s end. 
Similarly, Wilder School District, which experiences an average of 1 5 % turnover in staff each year, 
had a core group of administrators that is credited with sustaining the vision and strategic plan in spite 
of recurrent staff change. 

In contrast with these three districts, Lewis School District was on its third superintendent 
since Enhancement, and we were told that the second superintendent’s leadership and management 
style conflicted with the collaborative culture that had been established in the district during 
Enhancement and resulted in factions within staff. We also learned that collaboration and trust has 
not yet been fully rebuilt since the exit of the second superintendent three years ago, but that the 
district still has high hopes for reinstituting parts of the reform and district culture that had been in 
place when Enhancement ended. 

Consistency in building administrators also emerged as important. At Sitting Bull, the 
Enhancement contact told us that 70% of building administrators remain with the district since 
Enhancement, and these administrators continue to promote parent and community involvement with 
both new and veteran staff members. This informant told us that she believes the district has 
improved in including parents and community in school activities, and that inclusion of community 
“is a part of school culture” that is promoted by principals. Similarly, teachers reported to us that 
parent and community support and involvement is greater currently than it had been during 
Enhancement, in part because principals focus on this issue. 

The importance of stable district and school leadership for both implementing and sustaining 
reform is well-established (Berman & McLaughlin, 1977, 1978; Hargreaves & Fink, 2000; Bodilly, 
et al., 1998; McAdams, 1997). Yonezawa and Stringfield (2000) observed that principals where 
reform had been institutionalized had changed from assuming the role of a visionary leader to being 
a cooperative leader. That is, while it may be important for leaders to spearhead reform during its 
implementation, the supportive and cooperative role of principals may be more vital for sustaining 
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reform. When turnover in administration is unavoidable, the perpetuation of attention to and valuing 
of reform goals by new leaders is vital for continual progress. Research suggests that the careful 
recruitment of new administrators who are willing to consolidate and continue a reform that is in 
progress is necessary (Ross, 2000; Fink, 1999, 2000). In conclusion, although reform should not be 
identified with only one or a couple of people in the district’s administration, constancy in leadership 
as a whole - at both the district and school levels - had a dramatic influence on sustaining reform in 
the districts in this study. Furthermore, consistency in leadership ameliorated some of the potentially 
negative impact on reform progress that can be caused by high staff attrition at one site. 

Political context demands, pressures, and supportive activities. The influence of state 
policies and practices on school and district reform emerged as a complex issue in this study. First, 
we were told that the state’s focus on inputs (e.g., mandating the number of classroom hours spent 
on instruction in specific academic domains and the number of student-teacher contact days in the 
school calendar) as well as outcomes (i.e., student assessment results from the state’s norm-referenced 
testing program) is not conducive to innovation. More than one superintendent told us that the state s 
current requirements inhibit their districts’ continual progress of reform. 

On the positive side, state flexibility with respect to requirements was observed in the study. 
For example, Lewis School District sustained a state waiver since Enhancement that increased the 
number of professional development days and decreased the number of student-teacher contact days 
in that district’s school year. However, other districts had difficulty convincing their communities 
and school boards of the value of such a waiver. Thus, state policies and requirements hindered 
reform progress at sites, although flexibility in state policy benefitted at least one district in the study. 

Standards-based education and student test scores have recently gained in stature in the 
state’s accountability system. State standards are requiring districts to overhaul curricula and 
accelerate teacher familiarity with new learning objectives. Furthermore, because the state’s 
assessment is a norm-referenced test, teachers are being asked to present specific academic content 
at a much earlier age than in the past. The elementary school principal at Sacagewea had the 
following to say about influence of the new state standards on teachers: 

My fear is that, I'm thinking just with math as an example, that we used to remediate 
and enrich [as a result of Enhancement] and now we're on standards and we don't 
know really where that's taking us. You know, until we get to know the standards 
and say "OK. Here are the standards. We've taught them a year. Can we have our 
enrichment again? Can we have our remediation?” and work that whole program 
through ... And teachers are very— kind of uncomfortable with the standards. That's 
my perception...” 

At Lewis, a teacher commented that the state standards and testing program conflicts with the 
district’s previous reform agenda: “[The state’s governor] made [the standards] much more specific, 
you know. Much more teach this skill, keep this skill, teach this skill, and so that influence I think 
is very visible in our school, and maybe not as much in our school as other schools, but you can tell 
the difference. So it’s made a difference.” 
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In response to state testing practices, one district has contracted with a proprietary company 
for the use of assessment questions aligned with the state’s testing program. At that district, the 
superintendent reported a positive side of the new accountability focus in the state. He said that 
pressure from the state creates the feeling of “let’s slow down and get it right.” This superintendent 
valued the state’s focus on student test performance, though he acknowledged that it would be more 
appropriate for the state to use a criterion-referenced test than its current norm-referenced exam. 

Representatives from two districts mentioned that the state’s support and funding of 
innovation decreased drastically after the Enhancement Initiative ended, and that the absence of 
continued state support interfered with continuing reform. Thus, state support for innovative practices 
by school districts is another way in which political context can influence the continuation of reform. 

Preservice education was credited by some teachers with having sustained reform in the area 
of technology. That is, new teachers from local colleges and universities are coming into the school 
districts with capacity in the area of technology that supports the district’s reform goals in that area. 
Thus, preservice education is another arena in which political context (in part through what states and 
accrediting agencies support and require of institutions of higher education) influences district 
capacity to sustain reform. 

The local political context also greatly influences the sustaining of reform (Elmore, 1996; 
Fullan, 1 999). Communication with and support from the community proved vital for sustaining parts 
of the reform in these districts. For example, at Lewis, it was the support of the community and 
school board that maintained the weekly early student release time that had been so important to that 
district’s reform. Communicating the importance of that professional development time for teachers 
to the community and parents was essential for gaining approval of the district schedule that releases 
students for one afternoon a week. In several of the other districts we heard that it was the 
community, and consequently the school board, that placed a roadblock to having student release time 
implemented even though teachers and administrators realized the potentially powerful impact of 
additional planning time for teachers. 

Also underscoring the importance of school-community relations for sustaining reform was 
an example of a district in which conflictual relationships derailed reform for several years. The 
Sitting Bull superintendent told us the following: 

A huge public relations problem was an issue here ... That absorbed the teachers in 
this district, the administrators in this district for at least two full years, until you 
know, then things changed and they've gotten back ... we're kind of back on the way 
it used to be, but it was a kind of a hard time. And [Enhancement] ends and this 
other [public relations problem] begins, so that's been a little bit of a struggle for the 
district and for the staff. Teachers, now that they're out of that, they're kind of 
looking forward, saying, hey, let's get back to the [Enhancement]. 

Thus, maintaining communication and positive relationships with local communities and 
constituencies is crucial for sustaining progress in a district, because without community 
understanding of schools’ work, supportive funding and structures may be removed and teacher 
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morale may consequently decrease. The Sitting Bull example also suggests that conflict between 
schools and communities diverts energy away from the education system’s primary work of teaching 
students. 



Summary and Conclusion 

In summary, the Enhancement Initiative districts reported sustaining changes that were 
central components of the reform: namely, the use of research-based instructional practices, school- 
community collaboration and involvement, and the use of instructional technology. Furthermore, the 
changes that the Enhancement Initiative districts had implemented and sustained for nine years were 
less likely to be observed when Comparison districts were examined. 

Many factors influenced the continuation of changes made during Enhancement. First, 
human capacities that were integrated into routine activities were more likely to be sustained. 
Observations of positive results from those routinized practices and the training of new staff 
supported the ongoing integration of new activities. Second, the establishment of school and district 
cultures of innovation and collaboration focused on instructional goals supported continual progress 
toward reform. Changes in culture occurred during the reform initiative, and these cultural changes 
were so ingrained at these sites that reform was continued, in part, because of the cultural norms that 
had been adopted and relationships that were forged. Third, district structures helped to sustain 
reform. For example, new district positions, reallocations of resources, and new policies promoted 
continuation of activities such as new curriculum methods. Fourth, consistency in district and school 
leadership supported continuation. A corollary to this finding is the importance of recruiting, 
developing, and selecting new administrators who can perpetuate a district’s agenda (Ross, 2000; 
Fink, 2000). Fifth, the influence of political context can not be ignored. Although districts often 
express feeling out of control on this issue, the flexibility of state policies can sometimes be used to 
a district’s advantage, and the nurturing of relationships with a district’s constituencies can promote 
public support. 

Additionally, these five areas of influential factors should be considered in tandem. For 
example, sustaining reform through structures while ignoring culture has resulted in sustaining reform 
elements that are no longer effective at one of this study’s districts. Similarly, neglecting a district’s 
local political context while focusing on reform goals led to strained public relations that undermined 
staff morale in one district. It is likely that all five of these areas require attention to sustain continual 
improvement. Indeed, it seems possible that the comprehensive scope of this reform initiative, 
accompanied by local design and implementation of reform practices, supported the continuation of 
changes to the education system. That is, by addressing several of the capacities that might be needed 
for lasting school improvement (Stoll, 1999) during the initiative’s three years, it is likely that the 
design of the reform itself contributed to the sustained change observed. In future research, the 
contributions of work in the five areas presented here might be more closely examined for their 
contributions to sustained reform. 

The methodology for data collection in this study - namely, interviews with state department 
of education representatives, district administrators and teachers - puts limits on the ability to validate 



these conclusions. Participants in this study reported experiences and events, and it is likely that 
biases in memory and perception affected the study’s data. Measures were taken to remove some bias 
from the data, such as including multiple sources of information and matched comparison districts. 
Nonetheless, the study’s use of retrospective reports from individuals as its main source of data 
constrained both the type and objective validation of information that was collected. By corroborating 
our findings with similar research, we believe that the findings and conclusions made here are 
reasonable and accurate. Other research reports additional factors that promote continued reform 
(such as the use of multiple types of data or evaluation results), and therefore it is reasonable to expect 
that the five factors presented above are not a complete list of important considerations that should 
be attended to by districts and schools planning to continue reform. As mentioned early on in this 
paper, distinct settings are likely to require distinct strategies, perhaps even different strategies at 
different times, to influence the maintenance of change. 
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State Department Staff Interview 
Sustaining Complex Education Reform Project, McREL 



Name: 

Position: 

Interviewer: 

Date: 



1 What is the state currently doing to support district and school level reform? Probe on 1) 

standards and documents supporting standards-based instruction (e.g., curriculum 

frameworks), 2) professional development, 3) assessment, 4) accountability, 5) resources. 

2 What do you see as the major effects, impacts, and changes resulting from the Modernization 

Initiative on participating districts? 

(Probe with these areas if not already addressed:) 

a. Capacity building in terms of instruction (around OBE, technology, or what?) 

b. Development of infrastructure within the state for these districts (e.g., 
partnerships with TIE, State Department of Education, etc.). 

c. Structural changes, such as adding block scheduling or teacher planning time. 

d. District management policies, such as a movement toward site-based 
management that is data-driven 

e. Assessment policies to support reforms (e.g., use of waivers, alignment of 
curriculum and assessment policies, creation of new ones) 

f. Fostering good relationships within the district 

g. Fostering community support and involvement (e.g., businesses, parents, local 
foundations, etc.) 



3 . 



Do you think education reform is more sustained in some Modernization Initiative districts 
or schools than in others? As you think about those districts, what differences among the 
districts or schools do you think contribute to their ongoing reform progress? 



4 . 



For each of the following districts, would you please rate their degree of “reform” toward 
being an aligned, systemic education system on a scale of 1 to 10. A rating of 1 is not yet 
beginning reform; 4 is much reform having taken place in theory and policies but not 
practices, such as having adopted learning outcomes, but not implementing them; 7 is reform 
that is implemented in policies and practice but not by the entire education system; and 10 
is reform that involves a shared vision of student learning and outcomes that is reflected in 
the policies and practices of all teachers, administrators, and even external partners? 



1 . Lewis 


1 2 


3 


4 


5 


6 


7 


8 


9 


10 


2. Washington 


1 2 


3 


4 


5 


6 


7 


8 


9 


10 


3. Wilder 


1 2 


3 


4 


5 


6 


7 


8 


9 


10 


4. Lincoln 


1 2 


3 


4 


5 


6 


7 


8 


9 


10 


5. Sitting Bull 


1 2 


3 


4 


5 


6 


7 


8 


9 


10 


6. Jefferson 


1 2 


3 


4 


5 


6 


7 


8 


9 


10 


7. Sacagewea 


1 2 


3 


4 


5 


6 


7 


8 


9 


10 


8. Roosevelt 


1 2 


3 


4 


5 


6 


7 


8 


9 


10 



5. For each of the Modernization districts, could you recommend a contact person for us to 
interview about the Modernization Initiative’s immediate and long term effects. This person 
might be someone who has left the district but is still familiar with the district’s progress. 
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Modernization Contact Interview 
Sustaining Complex Education Reform Project, McREL 

District: 

Name: 

Position: Number of years in position: 

Interviewer: Date: 

1. Would you please describe your involvement with School District during the district’s 
participation in the Modernization Initiative (i.e., between 1990 and 1994)? 

2. Are there current district policies and practices that you believe are continuations or outgrowths 
of capacities that were enhanced through Modernization? (For each, how was the practice 
sustained? Where did support - financial, administrative - come from? What effect is this 
practice/policy having on the district/school now?) 

3. Are there any practices or policies during Modernization that should have been continued 
because of their effect and impact but were not continued? Why weren’t these activities 
continued? 



4. What do you see as the most positive long term effect of the district’s participation in the 
Modernization Initiative? 



5. Have there been any negative effects resulting from the district’s or schools’ participation in the 
Modernization Initiative? 



6. What did the district do to continue its progress after the Modernization Initiative ceased? Were 
there any specific strategies or policies that you believe contributed to the continuation of 
education reform after the Initiative? 
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District Superintendent Interview 
Sustaining Complex Education Reform Project, McREL 



District: 

Name: 



Position: 



Interviewer: 



Number of years in position: 
Date: 



1. What are the district’s major priorities this year? (Probe why each is a priority.) (If there is 
documentation listing these goals, e.g., a strategic plan, collect it.) 



2. What is the district doing to address or accomplish those priorities? How successful have these 
efforts and changes been? 

3. What challenges does the district face in addressing these priorities? 

4. Does the state have goals that coincide with the district’s priorities? How do state activities help 
or hinder the district’s progress? 

5. What is your district is doing to support learning more specifically? (For each, probe what 
support was received to implement changes, what the process of implementing changes is, how 
far activities have progressed, and what their effects have been.) 

[Probe with the following topics, only if necessary for thoroughness .] 

a. Identifying student learning and development goals. 

b. Professional development opportunities for teachers and administrators (including 
planning time for teachers and team teaching, for example). 

c. Other capacity building strategies (e.g., changes in staff assignments or responsibilities, 
fostering of relationships within and across the district). 

d. Assessment of students, evaluation. 

e. Accountability of schools and teachers. 

f. Addressing needs of students not meeting learning goals. 

g. Accrual and use of resources. 

h. Changes in district management policies (e.g., toward site-based management). 

i. Involvement of the community and/or businesses. 
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j. Partnerships or contracts with external organizations or agencies. 

k. New programs or instructional practices or changes in existing ones. 

l. Support use of technology in instruction. 

m. Any other changes or initiatives you wish to mention? 



6. When you think about the continual improvement that your district is engaged in, what 
district capacities, resources, policies, or practices do you believe have contributed to 
accomplishing ongoing continual improvement? 

Anything that has hindered the district’s path to continual improvement? 

7. [In primarily Native American districts ] How does the district address tribal culture in schools? 
[Only ask the following question of Modernization Initiative districts:] 

8. What role has the state’s Modernization Initiative, in your experience, played in the district’s 
reform process? 

b. Are there capacities and avenues to resources that you think your district developed 
during the Modernization Initiative that continue to support outcomes-based education 
today? 

c. How do you see those capacities contributing to continual improvement of the district 
and its schools? 

d. What support could you use from the state or other sources to help you sustain reform? 
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District Interview Protocol 

Curriculum, Instruction, Assessment, and/or Professional Development Specialists 
Sustaining Complex Education Reform Project, McREL 

District: 

Name: . 

Position: Number of years in position: 

Interviewer: Date: 

Note: If the interviewer feels that the district’s priorities have not been completely communicated 
from the Superintendent interview, ask the following two questions: 1 . What are the district’s 
major priorities this year? (Probe why each is a priority.) 2. What is the district doing to 
address these priorities? 

A. District efforts to implement outcomes-based education. 

1 . For each of the following areas, what is the district doing? (Probe to determine the 

nature of the activity, how far it has progressed and effects.) 

a. Developing and adopting a set of expected learning outcomes that should be 
expected of all students. (Who was involved? How were they agreed upon? For 
what grade levels do they exist?) 

b. Curriculum and instructional policies, programs, models, or practices and scaling 
up of new or modified ones, including use of technology in instruction 

c. Development of infrastructure (for example, changes in roles, responsibilities, and 
working relationships of district and school staff and activities to coordinate district 
and school reform efforts throughout the district) Who is in charge of these systems 
of support? 

d. Professional development activities and resources (e.g., partnerships with RELs, 
IHEs) 

e. Allocation and leveraging of state/local funds and obtaining grants or other external 
funds at the district level 

f. Assessment policies and evaluation to support reforms (e.g., alignment of 
curriculum and assessment policies, creation of new ones) 

g. Community support and involvement (e.g., businesses, parents, local foundations, 
etc.) 

h. Supporting site-based decision making regarding instructional and resource issues. 

i. Any changes in other areas. (Specify) 
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2. When you think about the continual improvement that your district is engaged in, 
what district capacities, resources, policies, or practices do you believe have 
contributed to accomplishing ongoing continual improvement? 

Anything that has hindered the district’s path to continual improvement? 



Effects of Changes. 

3. Teachers 

a. How have teachers responded to district changes? 

b. Do you have any evidence of changes in (i) teacher beliefs or (ii) instructional 
practice? (Obtain statistics if available.) 



4. Students 

a. What has been the impact on students of changes in programs, services, or 
instructional practices? (Any evidence of these changes in test scores, attendance 
rates, attrition rates, etc.? 

b. Is the district doing anything to assist students who are not achieving the identified 
learning outcomes? 



5. Try to characterize the amount of progress the district has made in implementing an 
outcomes-based education system, on a scale of one to five, in each of the following 



areas: 






12345 


a. 


A majority of teachers understand what outcomes-based reform is and 
the instructional implications. 


1 2 3 4 5 


b. 


A majority of teachers have aligned student outcomes with 
curriculum and assessment. 


12345 


c. 


Alignment of professional development to support implementation of 
outcomes-based education. 


1 2 3 4 5 


d. 


A majority of teachers have begun to change instructional practices 
to reflect identified student outcomes. 


1 2 3 4 5 


e. 


A majority of teachers are using student performance data to make 
instructional decisions. 



[Only ask the following question of Modernization Initiative districts:] 



6. What role has the state’s Modernization Initiative, in your experience, played in the 
district’s reform process? 

1 . Are there capacities and avenues to resources that you think your district developed 
during the Modernization Initiative that continue to support outcomes-based 
education today? 

2. How do you see those capacities contributing to continual improvement of the 
district and its schools? 

3. What support could you use from the state or other sources to help you sustain 
reform? 

7. Lastly, are there any recent accomplishments or changes that your district is particularly 
proud of that you could share with me? 
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School Principal Interview 

Sustaining Complex Education Reform Project, McREL 

District: 

Name: 

Position: Number of years in position: 

Interviewer: Date: 

Vision 

1 . What are your district’s major priorities this year? (For each, probe why.) 

2. What are your school’s major priorities this year? (For each, probe why.) 

3. What progress have you made towards these priorities? 

4. Do your school goals coincide with the district’s? With the state’s? Why or why not? 

School Capacity 

5. Do you evaluate the capacity or ability of people in your school to carry out instruction that 
will lead to improved student learning and achievement? 

Where do you see the major needs? 

What is being done by the state and district to meet these needs? Who is providing the 
support? Are these activities adequate? 

6. What kinds of capacity building activities does your school staff participate in — these can 
include teacher planning time and study groups as well as organized professional 
development activities? How widespread is involvement? Why have some (teachers, 
schools) participated but others have not? What would it take to get additional people to 
participate? 

7. Are there any particular changes underway in the design and delivery of instruction for 
students who are not achieving expected learning outcomes? Describe. 

8. Are there ways in which the community, local businesses, or other partners are involved in 
the school’s work? What have the effects of that involvement been? How has involvement 
been sustained over time? 
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School-level management 

9. How much discretion do schools have in making decisions regarding the following areas? 

a. . Curriculum development, coverage, and pacing 

b. Selection and use of instructional materials 

c. Use of resources 

d. Staffing (e.g., numbers of kinds of staff) 

e. Professional development activities 

f. School schedule 

g. Other decisions regarding school structure and instruction 

10. Please describe the process that would lead to changes affecting instruction, such as a 
change in scheduling? (Probe for whether and how teachers are involved in this process.) 

1 1 . To what extent are decisions made at the school level based on research or available data? 
(On a scale of 1 to 5, where 1 is never, and 5 is always.) 



12. When you think about the continual improvement that your school is engaged in, what 
district and school capacities, resources, policies, or practices do you believe have 
contributed to accomplishing ongoing continual improvement? 



13. Lastly, are there any recent changes or accomplishments that the school is particularly proud 
of that you would like to share with us? 
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Teacher Group Interview Protocol 

Sustaining Complex Education Reform Project, McREL 

Have teachers introduce themselves, with their classroom grade level. Ask teachers to each address 
all of the questions. 



1. School Culture. 

Can you please tell us how you feel about being a teacher in this school? For example, to what 
extent do you feel supported by your coworkers, that there are realistic or unrealistic demands 
made of you, that the community is concerned about student learning, and so forth? 

2. Instructional issues. 

We would like to get an understanding of what are current issues you face in educating your 
students. Could you please tell us what are current instructional issues for you, and what you, 
the school, and the district are doing to address these issues? What effects are these changes 
having and is it enough? (If necessary use probes such as: assessment, linguistic and cultural 
diversity, low performing children, state testing, etc.) 

Please tell us a little about how you are using technology in instruction, if at all. 

3. Professional Development. 

We would like to Find out what kinds of professional development activities you are engaged in, 
and what you see as the effects of those professional development endeavors. Professional 
development can mean technical assistance by external or district staff, or activities such as 
shared planning time for teachers or teacher study groups. (If necessary, probe for what help 
teachers are getting related to students achieving learning goals.) 

(For ongoing professional development, inquire about how long it has been in place and what 
the teachers believe has sustained the activity.) 

4. Teachers as Leaders. 

We would like to get a sense of how involved teachers are in making decisions that affect 
curriculum and instruction in this school and district. Would you please tell us about your 
involvement specifically, and the involvement of teachers in general in making decisions and 
policies that affect instruction, and if teachers are involved, can you talk a little more about how 
decisions are made? Examples of activities that teachers might participate in include (1) 
constructing standards documents, (2) constructing curriculum frameworks and aligning them 
with standards, (3) identifying professional development needs, (4) school decision-making 
teams. (Probe for what support is needed to get teachers more involved if this is an issue.) 
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5. Community and Business Involvement in Education. 

What are you, the school, and the district doing to involve the community and nearby businesses 
in education? (In Shannon and Todd, how are the tribal communities involved in the education 
system?) (Probe for involvement in decision making, if necessary.) 
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